Collective dynamics in sparse networks 
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The microscopic and macroscopic dynamics of random networks is investigated in the strong- 
dilution limit (i.e. for sparse networks). By simulating chaotic maps, Stuart-Landau oscillators, 
and leaky integrate-and-fire neurons, we show that a finite connectivity (of the order of a few tens) 
is able to sustain a nontrivial collective dynamics even in the thermodynamic limit. Although the 
network structure implies a non-additive dynamics, the microscopic evolution is extensive (i.e. the 
number of active degrees of freedom is proportional to the number of network elements). 
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The organization of dynamical phenomena on different 
scales is a general property of systems out-of-equilibrium, 
such as those encountered in plasma physics, turbulence, 
and neuroscience. The simplest instance of this hier- 
archical organization is the spontaneous emergence of 
collective behaviour out of a microscopically chaotic dy- 
namics, a phenomenon reminiscent of equilibrium phase- 
transitions. The first studies of collective dynamics con- 
tributed to uncover time-dependent macroscopic states 
with different degrees of complexity in mean field models 
[3, Q and in spatio-temporal chaotic models Q . 

Complex networks provide an even more interesting 
setup for the study of macroscopic phases, since this 
is the typical structure of many non-equilibrium sys- 
tems. Most of the studies of network dynamics have 
been so far devoted to the characterization of synchro- 
nized regimes 4, 5], where the single oscillators evolve 
in a coherent way. However, some preliminary studies, 
especially of neural networks with stochastic noise Q, 
have shown that self-sustained macroscopic oscillations 
can spontaneously arise also when the single elements 
evolve in a seemingly uncorrelated way. Altogether, the 
emergence of collective dynamics has been investigated in 
the presence of various ingredients such as delayed inter- 
actions, diversity of the single elements, time-dependent 
synaptic connections In particular, it is known that 

disorder may give rise to an extremely rich macroscopic 
scenario: this is indeed the framework where glassy phe- 
nomena have been uncovered [Io| and a highly irregular 
dynamics observed in neural networks [111 ]. 

In this Letter, we study several random networks to 
clarify the role played by the (in-degree) connectivity K 
(i.e. the number of incoming connections per node) on 
the onset of collective motion. It is convenient to dis- 
tinguish between two classes of systems fl2| : massive 
networks, where K is proportional to the network size 



N; sparse (or strongly diluted) networks, where K <^ N , 
and specifically K is independent of N as N —> oo. Lat- 
tice systems with short-range interactions belong to the 
latter class. While it is not surprising to observe the on- 
set of a collective motion in massive networks, it is less 
obvious to predict whether and when this can happen in 
sparse ones. In a model of leaky integrate-and-fire (LIF) 
neurons, it has been shown that a finite connectivity can 
sustain a partially synchronized regime 13[. Here, we 



show that the emergence of a collective dynamics above 
a finite critical connectivity K c is a general and robust 
property of sparse networks of oscillators. Since K c turns 
out to be of the order of a few tens in all models we have 
investigated, macroscopic motion appears to be rather 
ubiquitous and possibly relevant in the context of neural 
dynamics. In our simulations, we have typically assumed 
that all nodes are characterized by the same connectiv- 
ity K, but we have verified that the same scenario holds 
assuming a Poissonian degree distribution with average 
connectivity K, as in Erdos-Renyi graphs. 

Finally, we analyse the microscopic dynamics, irrespec- 
tive of the presence of the macroscopic phase, finding that 
it is always extensive (the number of unstable directions, 
as well as the power contained in the principal compo- 
nents, is proportional to the network size). This property 
is highly nontrivial, as the network dynamics is non addi- 
tive (it cannot be approximated with the juxtaposition of 
almost indepedent sub-structures, see below). This is at 
variance with globally coupled systems, which exhibit a 
non-extensive component in the Lyapunov spectrum [lij ] . 

More specifically we study three classes of dynamical 
systems: (i) units that are chaotic by themselves (logistic 
maps - LM); (ii) units that may become chaotic as a 
result of a periodic forcing (Stuart-Landau oscillators - 
SL); (iii) phase-oscillators that cannot behave chaotically 
under any forcing (LIF neurons). 
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Coupled maps. The dynamics on a network of N cou- 
pled logistic maps (LM) is denned as 
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x n +i(i) = (1 - g)f(xn(i)) + gh n (i) 



(1) 



where x n (i) represents the state of the ith node (i — 
1, . . . , TV) at time n, the logistic map f(x) = ax(l — 
x) rules the internal dynamics, and g is the coupling 

strength. Finally, h n (i) = {l-/K)Y,f=i S ijf( x n(j)) de- 
notes the local field, where is the connectivity ma- 
trix: Si. 



otherwise Sij = 
average 

{hl)-{K) 2 ) 



1 if an incoming link from j to i is present, 
0. It is convenient to introduce the 
field h n and its standard deviation {a\ = 

n . 

In Fig. [TJ <Jh is plotted versus the connectivity K for 
a = 3.9, g = 0.1 and increasing network sizes. For low 
connectivity, an is quite small and decreases as 
with the system size (see left inset), i.e. the deviation 
from zero is a finite-size effect. Above K c ~ 60, er/j as- 
sumes finite values, independently of the system size, sig- 
naling the onset of a collective dynamics. In fact, the 
right inset in Fig. Q] reveals nontrivial collective oscilla- 
tions for K = 500 and TV = 20, 000 (we have verified 
that the thickness of the "curve" does not decrease upon 
increasing the network size - data not shown) . The phase- 
portrait is analogous to that previously obtained in glob- 
ally coupled maps This indicates that the evolution 
of a sparse network reduces, for K — > oo, to that of its 
corresponding mean-field version. What is new and a 
priori non obvious is that a finite and relatively small 
connectivity suffices to sustain a macroscopic motion. 

As for the evolution of the single units, the most ap- 
propriate tool to investigate the microscopic dynamics is 
Lyapunov analysis. In Fig. [5£i,b we can see that both 
below and above K c the dynamics is characterized by 
extensive high-dimensional chaos 

nana 

, since the spec- 
tra of the Lyapunov exponents (LE) {A^} collapse onto 
one another, when they are plotted versus the intensive 
variable i/N [l8j]. In the inset, one can appreciate that 
the convergence occurs also for the largest LE, at vari- 
ance with the non-extensive behavior, recently detected 
in globally coupled networks 
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In a sparse network, the field h n (i) fluctuates with i, no 
matter how large the network is, since h n (i) is the sum 
of a finite number of contributions. One way to char- 
acterize its variability is by determining the covariance 
exponents (CE), i.e. the eigenvalues fii of the matrix 
Qj = {5h n (i)Sh n (j)) - (Sh n (i))(Sh n (j)), where Sh n (i) = 
h n {i) — h n . In ID spatial systems with periodic boundary 
conditions such an approach would correspond to deter- 
mine the spatial Fourier spectrum. In this case, since 
there is no "wavelength" to refer to, it is natural to order 
the eigenvalues from the largest to the smallest one. The 
results for different network sizes are plotted in Fig. [2] 
versus i/N, (panel c and d refer to K values below and 
above K c , respectively). The good data collapse confirms 
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FIG. 1: (color online) Model LM. Standard deviation of the 
mean field, Oh (averaged over 5 realizations of disorder) ver- 
sus K for TV = 5,000 (red) circles, N = 10,000 (green) 
squares, N = 20,000 (black) triangles, and TV = 40,000 
(blue) diamonds. The upper inset shows the rescaled ah at 
low K. In the lower inset the return map of h n for K = 500 
and TV = 20, 000. Symbols not connected by lines refer to 
K — 2N S + kfi (7V S = 40 is the number of nearest neighbours 
and kn of the random links) for TV = 5, 000 (violet) crosses, 
TV = 10, 000 (magenta) stars and TV = 20, 000 (red) plus. 



the extensivity of microscopic fluctuations. In both pan- 
els, [ii has been rescaled by to emphasize extensivity 
in yet a different way; in fact, as the local field is the sum 
of K contributions, its variance is expected to be on the 
order of 1/K. 




FIG. 2: (color online) Model LM. LE spectra are reported in 
(a) for K = 10 and TV = 100-200-500-1, 000, and in (b) for 
K = 80 and TV = 200-500-1000. CE spectra (rescaled by K) 
are reported in (c) for K = 10 and TV = 200-400-800-1, 600, 
and in (d) for K = 100 and TV = 800 - 1, 600 - 3, 200. In the 
insets a zoom of the largest values is shown. In this and in the 
following figures the arrow direction indicates data obtained 
for increasing system sizes. 

Stuart-Landau oscillators. The second model we have 
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analysed is a network of Stuart-Landau oscillators (SL), 
— Wi - (1 + ic2)\wi\ 2 Wi +g(l + ici)(Wi - Wi) , (2) 



where Wi 



is a complex variable, and Wi = 
Wj represents the local field (Sij is de- 
fined as before). Since the local variable is a complex 
number, it is convenient to introduce the global mean 
field W(t) = \ W\ (where | • | denotes the modulus opera- 
tion), which essentially coincides with the Kuramoto or- 
der parameter [19|. The data reported for (W) in Fig. [3^ 
reveals the discontinuous emergence of some form of syn- 
chronization (at least for our choice of the parameter val- 
ues, ci = —2, c 2 = 3 and g = 0.47 [III)- More precisely, 
there exists a finite parameter range (55 < K < 85), 
where, depending on the initial conditions Q, (W) may 
either vanish or take a finite value. A more precise char- 
acterization of the colletive phase can be obtained by 
looking at the variance of the order parameter, namely 
<r 2 w = (W 2 ) - (W) 2 . The data in Fig. [3b reveals that the 
discontinuous transition is accompanied by the birth of 
temporal fluctuations which increase with the connectiv- 
ity. As shown in the inset of Fig. [3Jd, the global attractor 
exhibits an irregular dynamics. Moreover, it is remark- 
able that the attractor is qualitatively different from the 
one found in the mean- field version of the model 2l| . 
which is centered around W ~ 0.35. 

As for LM, we have verified that both below and 
above the transition region, the microscopic dynamics 
is chaotic. In both cases, there is a clear evidence of 
a convergence towards an asymptotic LE spectrum (the 
LE spectra for K — 10 are reported in Fig. Analo- 
gous conclusions can be drawn from the CE spectra (see 

Fig. EH). 

Leaky integrate and fire neurons. Finally, we have con- 
sidered LIF pulse-coupled neurons. They are among the 
most popular and yet simple models used in computa- 
tional neuroscience, the field where understanding the 
onset of collective motion is likely to have the deepest 
impact. The evolution equation for the membrane po- 
tential Vi, writes as ii = a — Vi+ gei, where the local field 
e,i satisfies the equation p3| 

e.i + 2aii + a 2 ; e 4 = — ^ S a ^ n )8(t - t n ) . (3) 

n\t„<t 

Whenever the membrane potential vi reaches the thresh- 
old Vi = 1, it is instantaneously reset to the value vi = 
and a so-called a-pulse is emitted towards the connected 
neurons (for more details see (23[). In this case, we in- 
troduce the mean field E — Hi and the corresponding 
standard deviation <je- The model has been simulated 
for a = 1.3, excitatory synaptic strength g — 0.2 and 
inverse pulse- width a = 9. For such values, it is known 
that in the global coupling limit, there exists a periodic 
collective motion accompanied by a quasi-periodic mi- 
croscopic dynamics 24]. For small K, the mean field 
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FIG. 3: (color online) Model SL. Average (a) and standard 
deviation (b) of W versus K for N = 10, 000 (red) squares, 
and N = 20, 000 (blue) triangles (averaged over 5 realizations 
of the disorder). The dashed lines in (a) show the region of 
bistability (see the text). The inset in (b) shows the macro- 
scopic attractor for N = 10, 000 and K = 800. (c) LE spectra 
for K = 10 and N = 100-200-400 . (d) CE spectra (rescaled 
by K) for K = 10 and N = 40 - 80. 



is constant in the thermodynamic limit, revealing a so- 
called asynchronous state, while above a critical value 
K c w 9, it oscillates periodically, as seen in the inset of 
Fig. [4^,. As in the previous systems, the Lyapunov anal- 
ysis reveals an extensive behaviour, including the initial 
part (see the two insets), where at finite-size corrections 
of order 1/N are detected. This is to be contrasted with 
the initial non extensive "layer" observed in globally cou- 
pled systems [3| • Fully extensive behaviour was already 
observed for the 0-neuron model in [2^]. Finally, ex- 
tcnsivity is confirmed by the orthogonal decomposition 
applied to the fluctuations of the local field ej (see the 
CE spectra shown in Fig. \$jp) . 

Discussion. By studying three different models, we 
find that a finite connectivity is able to sustain a non- 
trivial collective motion, as signalled by the appearance 
of finite temporal fluctuations of the mean field. This sce- 
nario emerges in LMs, whose entire Lyapunov spectrum 
is positive, as well as in LIF neurons that cannot behave 
chaotically as stand-alone devices, even when subject to 
an irregular forcing. The differences among the various 
models concern the nature of the transition (continuous 
in LMs and LIF neurons, discontinuous and hysteretic 
in the SL oscillators) and the complexity of the collec- 
tive phase that is periodic in LIF neurons but certainly 
higher dimensional in the other two models. It is de- 
sirable to trace back analogies and differences to some 
general properties of the single systems. In this perspec- 
tive, it is natural to assume that a finite connectivity acts 
as a mean field accompanied by an effective noise (due to 
the "statistical" differences among the fields seen by the 
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FIG. 4: (color online) Model LIF. (a) Standard deviation of 
the mean field, o\e, versus K for TV = 1,000 (black) circles, 
TV = 5, 000 (red) squares, iV = 10, 000 (green) triangles. The 
inset shows the macroscopic attractor for TV = 5, 000 and K = 
200. (b) LE spectra (in the lower inset a zoom of the largest 
values) for K = 20 and TV = 240-480-960. In the upper inset 
the maximum Lyapunov exponent, X max , versus N is shown, 
the (red) line represents the nonlinear fit X m ax = 0.0894 — 
2.3562/TV and the (green) dashed line marks the asymptotic 
value, (c) CE spectra (rescaled by K) for K = 20 and TV = 
200 - 400 - 800 - 1600. 



different nodes). Accordingly, one expects that a small 
K corresponds to a large noise and is thereby unable 
to maintain a global coherence, as indeed observed. This 
picture is, however, rather qualitative, since the collective 
motion of SL oscillators markedly differs from that one 
generated in the mean-field limit, even when K 800. 
Altogether, it is remarkable that a few tens of connec- 
tions consistently suffice to sustain a collective motion in 
such different environments. 

In order to shed further light on the phenomenon, 
we have added a spatial structure, by organizing the 
nodes along a line and adding finite-distance interactions. 
When only the latter are present, no collective motion 
can arise, because of the low dimensionality of the lat- 
tice Q. The mixed case corresponds to a "small- world" 
arrangement [26[ ; we have imposed that the connectivity 
is the sum of two contributions (K = 2N S + kji): a num- 
ber ha of random links and 2TVg links with the nearest 
nodes. The data reported in Fig. [T] for TV S = 40 shows 
that it is sufficient to add kp — 20 links to establish a 
collective dynamics; in other words, in the presence of a 
lattice structure, a lower number of long-range connec- 
tions may be necessary, although the overall connectivity 
is larger. This observation reveals that the network struc- 
ture plays a nontrivial role in determining the number of 
links that can sustain macroscopic motion. 

Next, we comment on the extensive nature of the mi- 
croscopic motion, a property that is much less obvious 



than one could think. In fact, the existence of a limit 
Lyapunov spectrum in regular lattices is the natural con- 
sequence of the additivity of the dynamics jltij . Imag- 
ine to use a hyperplane P to divide a spatial domain 
of size TV into two subdomains Si and S2 of size TV/2. 
The mutual interaction between Si and S2 is negligible 
as it affects only the interfacial region around P (it is 
a "surface" effect). As a result, the entire system can 
be approximately seen as the juxtaposition of two inde- 
pendent subsets. In the case of sparse networks, it is 
not even obvious how to split them in two minimally- 
connected components Si, S2. This problem goes under 
the name of graph bipartitioning; it is known to be NP 
complete, and, more important, the solution involves a 
number of connections that is proportional to TV itself, 
whenever K > 2 log 2 (2^, [H| . Therefore, the "interface" 
cannot be likened to a "surface" and the evolution is nec- 
essarily non-additive. Accordingly, the extensive nature 
of the microscopic evolution is due to subtle properties, 
that have yet to be clarified. 

Conclusions and open problems. In this Letter we have 
shown that a double-scale (microscopic/macroscopic) 
evolution is a generic feature of sparse networks. Alto- 
gether, the existence of a critical connectivity separating 
asynchronous from coherent activity is similar to what 
experimentally observed in neuronal cultures (29j . In the 
perspective of understanding the conditions for the onset 
of this behavior, it will be worth exploring the depen- 
dence on the coupling strength. In particular, in the 
weak coupling limit, it might be possible to develop an 
analytic treatment (as already done for the synchroniza- 
tion transition in LIF neurons 13[ ) , although some of the 
temporal complexity might be lost. 
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at the University of Aarhus (Denmark). 
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